This invention relates to a method and apparatus for converting an interlaced video signal to the deinterlaced or progressive scan video signal, and in particular to a method and apparatus which provides appropriate control to the effectiveness of the conversion.
Broadcast television signals are usually provided in interlaced form. For example, the phase alternate line (PAL) system used in Europe is made up of video frames comprising two interlaced fields. Each field comprises alternate lines of the frame. Thus, when the signal is applied to a display the first field will be applied to the odd numbered lines of the display followed by the second field being applied to the even numbered lines of the display. The frame rate, the rate at which frames comprising two interlaced fields are applied to a display is usually 50 Hz.
Progressive scan displays interpolate within the fields of each frame and sometimes between adjacent fields to provide data for the missing lines in each field, thereby converting each field to a frame and doubling the effective frame rate of the display. One of the problems when interpolating the missing lines of video fields is that of accurate detection of edges or contours marking variations in the visible information. U.S. Pat. No. 5,532,751 looks at the variation between pixels which are used to interpolate missing pixels to detect edges or contours. If the variation is below a threshold, the orientation of an edge is estimated and a new pixel is formed from the average of the pixels lying along the estimated orientation. If the estimate of edge orientation is unsuccessful then a new pixel is formed from the average of two vertically aligned pixels within a field. This technique can generate artefacts in pictures which have two or more pairs of pixels with high resemblance.
An improvement upon this method is disclosed in U.S. Pat. No. 6,133,957. In this, the variation between pixels or a set of pixels is computed to reconstruct edges or borders. Two variations with the lowest values are used and a reconstructed pixel is generated as a weighted average of the pixels used in the chosen variations.
Still a further improvement is set out in British patent no. 2402288. The solution presented here preserves vertical frequencies present in a frame which is being deinterlaced when accurate information on the position of an edge or border is not available.
All the techniques described above fetch input data from one instant of time only and search for the best match in vertically adjacent lines of a video field. They are referred to here as border reconstructers (BR).
One of the fundamental ideas behind a BR is the estimation of the correlation between two sets of pixels belonging to two vertically adjacent lines in a field at an instant of time.
FIG. 1 shows three representations of short sections of two adjacent lines in a video field. In the example given in FIG. 1, we see only the lines from the current field being used although one or more adjacent fields can also contribute to the interpolation used to the derivation of pixel data for the missing lines as can additional lines in the current field.
In FIG. 1, three different possible interpolations schemes are shown and correlations are evaluated for these. The middle scheme comprises correlation of the data in the pixels above and below the pixel to be reconstructed and correlation of data between pairs of pixels positioned immediately adjacent to this. A further possible interpolation is evaluated in the left-hand example of FIG. 1 by looking at the correlation between pixels on lines which pass diagonally sloping down to the right through the pixel being reconstructed. The same process with the opposite diagonals is shown in the right-hand example of FIG. 1.
The correlation between the data in the various pairs of pixels can be derived using the sum of absolute differences (SAD) or the mean square error, or other well-known statistical techniques. The sum of absolute differences and the mean square error are derived as follows:
      SAD    =                  ∑        i            ⁢                                            Ytop            ⁡                          [              i              ]                                -                      Ybot            ⁡                          [                              n                -                i                            ]                                                        MSE    =                  ∑        i            ⁢                        (                                    Ytop              ⁡                              [                i                ]                                      -                          Ybot              ⁡                              [                                  n                  -                  i                                ]                                              )                2            In the above formulas, Ytop and Ybot represent the luminance of the pixels in the lines above and below the pixel to be reconstructed in a field, and n is the number of pixels in each row. The luminance of a pair of pixels is involved in each single difference.
The graph on the right-hand side of FIG. 1 shows an example of SAD based procedure using five pixels only for each row and three correlations of symmetrically located sets of pixels, each set made up of the three pixel pairs. In practice, more pixels are involved in the computation to ensure greater accuracy. Preferably, between 7 and 30 pixels pairs are used.
If we use the SAD approach to comparing the values of pairs of pixels, then FIG. 1 leads to 3 SAD values. SAD 0, SAD 1 and SAD 2 which are shown graphically at the right-hand side of FIG. 1. This can be considered the correlation curve for the various possible interpolations. In many techniques, the interpolation scheme which gives the smallest difference in SAD or the smallest means square error (MSE) does not always produce the best quality final image. This is because the content of the image in the neighbourhood of the pixel being reconstructed can affect the SAD or MSE. For example, if there are a few thin lines passing close to the pixel to be reconstructed there is a risk that in reconstruction, the lines result in pixelation or flickering in the final image. In U.S. Pat. No. 6,133,957 and GB 2402288 this problem has been addressed by blending several relative minima in a correlation curve together and has also been approached by clamping the result using the values generated by the pixels directly above and below the one to be reconstructed. The problem with these approaches is that even though blending and damping reduce the effect of incorrect analysis of the correlation curve, they are affected by the incorrect starting point for the procedure.
We have appreciated that by modifying the correlation curve with an adjustment curve selected in dependence on the form of the correlation curve increases the likelihood of selecting the correct minimum value from the correlation curve. The adjustment curve is selected or altered in dependence on a confidence measure derived from the correlation curve data.
Preferably, the local minima for various portions of the correlation data are detected and the selection of an adjustment curve to combine with the correlation curve to generate ft most likely interpolation scheme to produce good results is made in dependence on the relative positions of minima in the correlation data.
Preferably the correlation data is divided into segments and local minima detected in each segment.